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SUMMARY 


The  research  performed  for  this  grant  over  the  past  year  involved  affine  iterated 
function  system  (IFS)  encoding  and  IFS  mixing  for  digital  images.  This  relates 
to  a  technique  of  Michael  Barnsley’s  for  generating  fractal  and  other  images  by 
randomly  iterating  affine  transformations  of  the  plane  into  itself.  By  this  tech¬ 
nique  an  image  is  both  generated  and  represented  as  the  long-term  probability 
distribution  for  a  2-D  or  3-D  Markov  chain.  The  encoding  involves  finding  an 
affine  “collage”  of  the  image,  whereby  it  is  identified  as  a  convex  combination  of 
affinely  scaled  versions  of  itself.  This  permits  some  remarkable  data  compression. 
The  mixing  involves  a  merging  of  IFS’s  so  as  to  produce  images  with  combined 
textures.  It  ties  in  with  the  encoding  in  that  a  broader  class  of  images  can  then  be 
efficiently  encoded,  and  there  are  more  degrees  of  freedom  in  the  encoding  search. 
The  mathematical  methods  used  involve  stochastic  optimization,  computational 
geometry,  the  Radon  transform,  dynamical  systems  and  ergodic  theory  for  Markov 
chains.  In  addition  the  proposer  studied  the  dynamics  of  discrete  IFS,  whereby 
orbits  are  truncated  so  as  to  always  land  in  centers  of  pixels.  Even  in  the  strictly 
contractive  case,  the  discrete  IFS  is  not  strictly  contractive,  and  uniqueness  of 
stationary  distributions  is  lost.  Nevertheless  it  was  shown  that  as  the  pixel  size 
goes  to  zero,  any  sequence  of  stationary  distributions  converges  weakly  to  the 
stationary  measure  for  the  true  IFS.  Finally,  the  proposer  animated  IFS  image 
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generation  so  as  to  produce  video  segments  representing  flows  of  images.  This  is 
to  be  used  for  animation  encoding,  whereby  a  dynamical  sequence  of  images  is 
encoded  as  a  time-dependent  IFS. 

STATEMENT  OF  WORK 

BASIC  IFS  MODEL:  The  basic  IFS  image  generation  algorithm  is  illus¬ 
trated  in  Fig.  1.  The  leaf  is  generated  as  follows.  Pick  any  point  XoeS?2-  There 
are  four  affine  transformations  T  :  x  *-►  Ax  +  b  listed  on  the  top  of  this  Fig., 
and  four  probabilities  p,  underneath  them.  Choose  one  of  these  transformations 
at  random,  according  to  the  probabilities  p, — say  7*  is  chosen,  and  apply  it  to 
Xo,  thereby  obtaining  Xj  =  7*X 0.  Then  choose  a  transformation  again  at  ran¬ 
dom,  independent  of  the  previous  choice,  and  apply  it  to  Xl5  thereby  obtaining 
X2.  Continue  in  this  fashion,  and  plot  the  orbit  {X„}.  The  result  is  the  leaf 
shown.  By  tabulating  the  frequencies  with  which  the  points  X„  fall  into  the  vari¬ 
ous  pixels  of  the  graphics  window,  one  can  actually  plot  the  empirical  distribution 

ELo  using  a  grey  scale  to  convert  statistical  frequency  to  grey  level.  The 
darker  portions  of  the  leaf  correspond  to  high  probability  density. 

The  following  is  a  brief  description  of  the  proposer’s  mathematical  model  for 
IFS  theory.  Let  G  be  the  semi-group  of  affine  transformations  g  :  x  ax  +  b 
from  — >  &m.  Let  peP(G),  the  collection  of  Borel  probabilities  on  G.  Given 

1 define  the  convolution  p  *  v  as 
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H*v{B)  =  J  v{g  1B)fi(dg) 


for  Borel  subsets  B  C  Equivalently  if  geG  is  distributed  like  /x,  if  Xt3?m  is 
distributed  like  v  and  if  g  and  X  are  independent,  then  /x  *  u  is  the  distribution 
of  gX..  Say  that  veP($lm)  is  /x- stationary  when  /x  *  v  —  v.  It  can  be  shown  that  if 
v  is  /x-stationary  then 

c  =  |J  sc 

giH 

where  C  and  H  axe  the  supports  of  v  and  /x,  respectively.  This  is  the  Collage 
Property.  Examples  like  Fig.  1  correspond  to  the  case  where  /x  is  an  atomic 
measure 


n({Ti})  =  pi,  \<i<N.  (*) 

In  this  case  the  stationarity  condition  becomes 


"(B)  =  £,pMT-'B) 
1=1 

and  the  Collage  Property  is 


C  =  (J  TtC. 

1=1 

This  is  illustrated  in  Fig.  2.  Observe  how  (7,  the  grey  leaf,  is  covered  by  four 
black  leaves — each  of  which  is  an  affine  copy  of  C. 
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Let  {<7„}  be  an  i.i.d.  sample  from  ft.  The  Markov  chain  associated  with  ft  is 


defined  by 


Xn+i  —  gn+ iXn,  n  >  0. 

When  ft  is  the  atomic  measure  given  by  (*)  above,  then  this  chain  is  precisely 
the  process  described  in  the  beginning  of  this  Section.  A  distribution  i/eP(9?m) 
is  stationary  for  this  chain  if  and  only  if  it  is  //-stationary.  The  proposer  showed 
in  [2],  [4]  that  if  ft  obeys  a  suitable  contractivity  condition,  then  there  exists  a 
unique  //-stationary  i/eP(3fm)  and  the  following  Law  of  Large  Numbers  holds. 

(LLN)  With  probability  one  the  empirical  distributions  Ylk=o  converge 
weakly  to  v. 

This  LLN  ensures  that  the  plot  of  the  orbit  of  any  single  trajectory  {X„}  will 
produce  the  desired  image. 

IFS  ENCODING  (work  with  J.-P.  Vidal):  There  are  two  types  of  IFS  im¬ 
age  encoding  techniques  under  development  today:  interactive  computer-aided 
encoding,  and  automated  encoding.  In  both  instances  the  objective  is  to  encode 
a  given  target  digital  image  so  that  it  can  be  re-generated  as  the  attractor  of  an 
affine  IFS.  For  the  interactive  encoding  one  sits  down  at  a  terminal  and  defines 
various  affine  transformations  geometrically,  with  the  aid  of  a  mouse,  by  identify¬ 
ing  images  of  triangles  or  rectangles  under  the  transformations.  Probabilities  are 
either  assigned  to  them  in  proportion  to  their  determinants  (area  factors),  or  else 
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user-specified.  Then  the  attractor  of  the  IFS  is  generated  and  overlayed  upon  the 
target  image.  Based  on  this  the  user  can  go  back  and  modify  his  transformations 
until  the  attractor  fits  well  with  the  target.  When  this  occurs  the  image  has  been 
encoded,  and  in  fact  the  coefficients  and  probabilities  for  the  affine  transforma¬ 
tions  constitute  the  code.  For  images  involving  roughly  ten  transformations  in  the 
IFS,  this  interactive  encoding  typically  takes  between  fifteen  and  sixty  minutes 
on  a  micro- Vax,  depending  on  the  accuracy  desired  (assuming  that  the  user  is 
familiar  with  the  Collage  Property). 

For  automated  encoding  a  target  digital  image  is  input,  and  the  affine  trans¬ 
formations  and  probabilities  for  the  IFS  are  all  internally  calculated.  The  error 
between  the  IFS  attractor  and  the  target  image  is  measured  by  the  Hausdorff 
distance,  and  the  automated  encoder  searches  over  affine  transformations  so  as  to 
minimize  this  error.  This  type  of  encoding  typically  takes  from  one  to  ten  hours 
on  a  micro- Vax. 

Algorithms  for  both  types  of  encoding  were  developed  and  coded  under  this 
grant,  and  they  are  currently  being  tested  and  optimized.  For  the  automated 
encoding  the  proposer  developed  two  algorithms — one  involving  extreme  points 
of  the  convex  hull  of  the  image,  and  one  involving  sequential  constrained  opti¬ 
mization.  The  former  was  described  in  the  First  Annual  Technical  Report,  and 
the  latter  goes  as  follows.  At  stage  i  solve  for  T,  by 
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MAXIMIZE  [area  (r,C  f|  Ri- 1 )  -  A  area^T.C  -  C)] 
affine  contractions  T, 
where  .ft,  are  the  residual  sets  fto  =  C, 

Ri  =  ft,_,  -  TiC,  i  >  1, 

and  A  is  an  adjustable  adaptive  parameter.  Furthermore  after  the  transformations 
Tu...,Tn  have  been  obtained  in  this  way  they  can  be  refined  on  successive  sweeps 
by  modifying  the  residuals 

R%- 1  *—  Ri- 1 U 

where 

£i  =  cf|r.c-U:ric 

is  the  essential  part  of  TiC;  namely,  those  points  of  C  covered  exclusively  by 
TiC.  This  sequential  approach  has  the  nice  feature  that  once  is  computed  the 
algorithm  automatically  stays  away  from  the  identity  map,  since  it  only  tries  to 
cover  the  residual  of  C. 

IFS  MIXING  (work  with  M.  Barnsley):  During  the  first  year  of  this  grant 
the  proposer  [2]  developed  a  mixing  algorithm  for  combining  several  image  tex¬ 
tures.  This  mixing  includes  both  IFS  condensation  [1,  Sec.  3.9]  and  recurrent  IFS 
[3]  as  special  cases.  The  analysis  of  the  mixing  was  only  carried  out  then  for  the 
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case  of  two  screens  (i.e.  the  mixing  of  two  images).  This  year  the  analysis  was 
performed  for  the  general  case  of  N-screen  mixing.  It  involves  the  asymptotics  for 
products  of  random  affine  maps  indexed  over  excursions  of  a  finite  state  Markov 
chain  from  a  fixed  recurrent  state.  The  mixing  process  is  ergodic  if  the  Lyapunov 
exponent  of  such  a  random  product  is  negative.  The  proposer  is  currently  working 
on  encoding  schemes  for  mixed  processes,  and  more  generally  for  images  which 
are  2-D  or  3-D  cross-sections  of  higher  dimensional  IFS. 

DISCRETE  IFS  (work  with  M.  Perrugia):  Set  up  window  coordinates  so 
that  the  centers  of  the  pixels  are  at  locations  (] tf'Hf)  f°r  integers  i,j.  Assume 
that  every  point  generated  by  the  IFS  is  sequentially  replaced  with  the  center  of 
that  pixel  in  which  it  lies.  The  resulting  discrete  Markov  chain  is  referred  to  as 
a  discrete  IFS.  If  the  true  IFS  is  strictly  contractive  the  discrete  IFS  need  not 
be.  Consider  for  example  the  1-D  IFS  consisting  of  the  single  map  T  :  x  *-»  §x. 
Suppose  M  =  10  and  consider  the  pixel  (0.05,  0.15)  which  is  centered  at  x  =  0.1. 
For  this  x  one  has  Tx  —  0.0667,  and  after  rounding  one  is  back  to  x  =  0.1.  Thus 
x  =  0,  ±0.1  are  all  absorbing  states  for  the  discrete  IFS  here.  The  proposer  has 
shown  that  if  the  true  IFS  is  strictly  contractive  then  as  M  ->  oo  any  sequence 
of  stationary  distributions  for  the  discrete  IFS  converges  weakly  to  the  (unique) 
stationary  distribution  for  the  true  IFS.  Currently  he  is  investigating  conditions 
for  ergodicity  of  the  discrete  IFS. 
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IFS  ANIMATION 


Suppose  one  has  two  IFS,  say  {7i,p,  :  1  <  i  <  N)  and  { I :  1  <  i  <  N}. 
Each  of  these  IFS  corresponds  to  an  image  and  one  can  set  up  a  continuous  flow 
of  images  from  one  to  the  other  by  interpolating  the  IFS  parameters.  This  is 
by  no  means  the  same  as  pixel  interpolation.  If  v  and  v'  are  p  and  //-invariant, 
respectively,  then  pixel  interpolation  would  amount  to 

v(t)  =  (1  -  t)v  +  tv\  0</<l. 

This  sort  of  interpolation  generally  results  in  blurred  intermediate  images.  The 
proposer’s  scheme  amounts  to  letting  i/(t)  be  the  //(t)-invariant  probability,  where 

p(<)  =  (1  -  <)p  +  <p',  0<*<1. 

This  always  produces  a  clear  and  distinct  sequence  of  intermediate  images. 

What  makes  IFS  animation  even  more  special  is  the  ease  with  which  one  can 
rotate,  scale,  change  perspective  or  vantage  point,  zoom  in  and  out,  or  perform  any 
affine  transformation  on  an  image.  Suppose  one  wants  to  apply,  say,  a  3-D  rotation 
R  to  the  image.  This  can  be  incorporated  directly  into  the  IFS.  Simply  replace  the 
original  transformations  Tt  with  the  composite  transformations  RT^R~l .  These 
composite  transformations  are  also  affine,  and  by  running  the  IFS  algorithm  with 
them  and  using  the  same  probabilities  as  before,  the  rotation  is  automatically 
built  in. 
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IFS  animation  is  a  highly  parallel  algorithm.  The  images  for  the  various  times 
t  in  the  flow  can  all  be  generated  in  parallel,  since  there  are  no  dependencies 
among  the  images  at  different  times.  Furthermore  the  same  sequence  of  random 
numbers  can  be  used  for  all  the  intermediate  images.  Figs.  3-5  are  some  snaps 
from  different  animations. 

The  next  step  is  IFS  encoding  for  animation.  Here  the  data  compression  ratios 
are  enormous,  since  the  encoding  of  two  “endpoint”  images  suffices  to  generate 
the  intermediates.  Furthermore  in  certain  respects  animation  encoding  is  easier 
than  still  image  encoding.  This  is  because  a  dynamic  sequence  of  images  often 
exposes  more  information  about  the  individual  still  images,  such  as  segmentation 
information.  Velocity  tracking  of  boundaries  and  key  features  of  an  image  can  be 
used  to  decide  where  to  position  the  temporal  IFS  linear  interpolation  points  (i.e. 
to  break  the  animation  up  into  “piecewise  linear”  video  segments),  and  how  to 
identify  the  images  as  IFS  mixtures.  Animation  is  potentially  the  most  exciting 
application  of  IFS  encoding. 

REFERENCES: 

1.  Barnsley,  M.F.,  Fractals  Everywhere ,  Academic  Press,  New  York,  1988. 

2.  Barnsley,  M.F.,  Berger,  M.A.,  and  Soner,  H.M.,  Mixing  Markov  chains  and 
their  images,  Prob.  Eng.  Inf.  Sci.  2  (1988),  387-414. 


9 


3.  Barnsley,  M.F.,  Elton,  J.H.,  and  Hardin,  D.P.,  Recurrent  iterated  function 
systems,  Constr.  Approx.  5  (1989),  3-31. 

4.  Berger,  M.A.  and  Soner,  H.M.,  Random  walks  generated  by  affine  mappings, 
J.  Thtor.  Prob.  1  (1988),  239-254. 

PUBLISHED  ARTICLES: 

Barnsley,  M.F.,  Berger,  M.A.,  and  Soner,  H.M.,  Mixing  Markov  chains  and 
their  images,  Prob.  Eng.  Inf.  Sci.  2  (1988),  387-414. 

Berger,  M.A.,  Computational  methods  for  two-phase  flow  in  networks,  in 
Liquid  Metal  Flows:  Magnetohydrodynamics  and  Applications,  H.  Branover 
et  al,  eds.,  Progress  in  Astronautics  and  Aeronautics,  Vol.  Ill,  AIAA,  Wash¬ 
ington,  DC,  539-572. 

Berger,  M.A.,  Images  generated  by  orbits  of  2-D  Markov  chains,  CHANCE 
2  (1989). 

Berger,  M.A.,  Encoding  images  through  transition  probabilities,  Mathl.  Corn- 
put.  Modelling  1 1  (1988),  575-577. 

Berger,  M.A.,  Random  affine  IFS:  mixing  anu  encoding,  to  appear. 

Berger,  M.A.  and  Soner,  H.M.,  Random  walks  generated  by  affine  mappings, 
J.  Theor.  Prob.  1  (1988),  239  254. 


10 


INTERACTIONS:  The  proposer  gave  the  following  presentations: 

(1)  AFOSR  at  Bolling  Air  Force  Base  on  Feb.  24, 1988  (host:  Dr.  A.  Nachman); 

(2)  NIST  in  Gaithersburg,  MD  on  March  29,  1989  (host:  Dr.  F.  Sullivan); 

(3)  NSF  in  Washington,  DC  on  March  30,  1989  (host:  Dr.  R.  Chin); 

(4)  Invited  talk  in  Michael  Barnsley’s  minisymposium  on  chaos  at  the  annual 
SIAM  meeting  in  Minneapolis,  July  10-15,  1988  (host:  Dr.  M.  Barnsley); 

(5)  Sixth  International  Conf.  on  Math.  Modelling,  held  at  Washington  Univ. 
in  Aug.,  1987; 

(6)  Invited  talk  for  the  seminar  run  by  the  image  processing  group  (Grenander, 
McClure,  Geman,  and  Gidas)  in  the  Division  of  Applied  Mathematics  at 
Brown  University  on  Feb.  15,  1989  (host:  Dr.  B.  Gidas); 

(7)  Invited  talk  for  the  seminar  run  by  the  Pittsburgh  Supercomputing  Center 
in  October,  1987  (host:  Dr.  R.  Roskies); 

(8)  Special  3-day  lecture  series  at  Allegheny  College,  April  10-12,  1989. 

PARTICIPATING  PROFESSIONALS:  The  proposer’s  research  on  image 
encoding  is  being  carried  out  with 

(1)  Jean-Philippe  Vidal  (Ph.D.  student,  Comp.  Sci.,  CMU-funded  by  AFOSR); 
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(2)  Bill  Eddy  (Prof,  of  Statistics,  CMU); 

(3)  Mario  Perrugia  (Ph.D.  student,  Statistics,  CMU); 

(4)  Michael  Barnsley,  John  Elton,  Jeff  Geronimo,  and  Ron  Shonkwiler  (Prof.’s 
of  Math.,  GA  Tech.). 

The  proposer’s  research  on  mixing  is  being  carried  out  with 

(1)  H.  Mete  Soner  (Prof,  of  Math.,  CMU); 

(2)  Michael  Barnsley  and  John  Elton  (Prof.’s  of  Math.,  GA  Tech.). 

Computing  support  has  been  provided  by  the  Pittsburgh  Supercomputing  Center 
(Cray  Y-MP/832  and  animation  equipment),  the  Statistics  Dept,  at  CMU  (micro- 
Vax  and  animation/camera  equipment)  and  the  Computer  Graphics  Lab  in  the 
School  of  Mathematics  at  GA  Tech.  (Masscomp  5600  series  and  Encore).  An 
article  about  the  proposer’s  work  on  IFS  image  processing  (written  by  science 
editor  Michael  Schneider)  will  appear  in  the  forthcoming  Projects  in  Scientific 
Computing  for  the  Pittsburgh  Supercomputing  Center  this  June. 
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Figure  2:  Collage  of  the  Maple  Leaf 


Figure  3:  IFS  Animation.  Scenes  from  a  “walk  in  parameter  space”  which  flows 
continuously  from  the  spiral  (a)  to  the  twin-dragon  (f). 


Figure  4:  Intermediate  images  in  an  image  flow  from  leaf  (a)  to  fern  (f).  The 
images  depend  continuously  on  the  IFS  parameters,  so  that  small  changes  m 
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Figure  5:  Successive  Zoom-In’s  of  the  Dragon.  The  larger  the  magnification  the 
more  points  get  filtered  out  of  the  IFS  simulation. 


